Skip to content

fix(capabilities): require explicit intent for named groups - #357

Merged
tt-a1i merged 4 commits into
openpi-dev:mainfrom
seekskyworld:fix/issue-303-capability-intent
Sep 12, 2026
Merged

fix(capabilities): require explicit intent for named groups#357
tt-a1i merged 4 commits into
openpi-dev:mainfrom
seekskyworld:fix/issue-303-capability-intent

Conversation

@seekskyworld

Copy link
Copy Markdown
Contributor

Problem

Closes #303. Bare subagent/workflow references in identifiers, discussion, or comparisons were treated as authorization under explicit discovery, causing optional tool groups to load unexpectedly.

Value

Prevents accidental capability activation from pasted prompts and ordinary development vocabulary while preserving explicit delegation/workflow requests.

Approach

Require imperative English request forms for named capabilities, retain existing Chinese request forms, and add regressions for branch/file references and comparisons. Negated and conditional clauses continue to fail closed.

Validation

  • node --test --experimental-strip-types tests/extensions/shared/capability-intent.test.ts (5 passed)
  • git diff --check (passed)

Impact

  • User-visible behavior: fewer false-positive capability highlights/loads.
  • Model-visible tools: explicit requests unchanged; bare references no longer load groups.
  • Runtime/lifecycle: no changes beyond intent classification.
  • Persisted config/data: none.
  • Compatibility/risk: prompts that relied on bare capability names must use an explicit request.

Signed-off-by: seekskyworld <djh1813553759@gmail.com>

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

需要改:祈使语境仍然太宽

去掉光秃的 subagent / workflow 保留字是对的。但 delegate 仍是:

\b(?:use|spawn|start)\s+(?:a\s+)?subagents?

「I use subagents in this repo」这种讨论句仍会授权。请再收紧到明确的委派/启动请求,并补这条负例。

@tt-a1i

tt-a1i commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

2026-09-09 对照(未合并、未跑 Session)。origin/main 0d17f45 与本工作区仍是保留授权词实现。按本 PR 正则本地复算:

仍会加载(#303 未解):我开发对应改代码子代理模块开始看子代理代码开个子代理接下来讨论子代理设计I use subagents in this repohow to use subagents。中文 (?:使用|用|启动|调用|来|开).{0,8}子代理 原样保留;use|spawn|start + subagent 仍吃讨论句。

会变成 inert(符合英文标识符目标):分支/文件/路径、Compare Subagent and Workflowsubagent, workflow、GitHub Actions 的 workflow.github/workflows/ci.yml

新漏开:Use multiple subagents.origin/maintests/extensions/capabilities/index.test.ts 仍断言这是显式委派。

CI 红在 biome format(for-of 写成一行),还没跑到会失败的 sibling 用例——本 PR 只改了 capability-intent 测试,没有改 ui.test.ts、reserved-word index.test.tsSETUP.mdREADME.mdtt-a1i 的 CHANGES_REQUESTED 仍然成立。

建议不要按当前 diff 合并。完整切片:拆保留词 + 不要用 use + name 补回讨论句 + 处理中文 / 前缀 + 同步三份测试和 SETUP/README。漏掉「只打了 subagent」应写成政策变更,不要用更宽的正则补回来。本轮不在工作区实现。

本地对照:docs/research/EXPERIMENT_KEYWORD_CLASSIFIER_2026-09-09.md

@tt-a1i

tt-a1i commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

2026-09-09 follow-up: exact test contract this PR is still missing. The 09-09 earlier comment listed leftover firings and files; it did not publish file + assertion + expected, or which SETUP/README sentences become false after a merge. Local record: docs/research/EXPERIMENT_CLASSIFIER_CONTRACT_2026-09-09.md. Still no Session / no pi install / classifier not implemented here.

Do not merge this diff. Smallest fail-closed slice remains: drop reserved words, do not replace with use+name, take / out of the Chinese verb list, move all three test files + SETUP/README together.

Tests this PR never added (must be [] / no load / no highlight)

tests/extensions/shared/capability-intent.test.ts — review sentence is still absent:

assert.deepEqual(capabilitiesRequestedByPrompt("I use subagents in this repo"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("we use subagents here"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("how to use subagents"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("users can use subagents"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("I use workflows in this repo"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("how to use workflows"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("我开发对应改代码子代理模块"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("开始看子代理代码"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("接下来讨论子代理设计"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("打开子代理面板"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("过来看看子代理"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("开个子代理"), []); // accepted miss; document in SETUP

Also missing identifier/CI locks beyond the three already added: feat/subagent-isolation, extensions/subagents/, edit .github/workflows/ci.yml, this is a GitHub Actions workflow, read workflows/SKILL.md, capability-intent.ts mentions subagent, and bare subagent / workflow / Workflows / SUBAGENT.

Polarity already wrong in this PR (these lock use+name, which CHANGES_REQUESTED rejected):

// PR currently expects delegate/workflow; fail-closed contract is []
assert.deepEqual(capabilitiesRequestedByPrompt("Use subagents to review this change"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("Use a subagent to review this change"), []); // PR *added* this as a positive
assert.deepEqual(capabilitiesRequestedByPrompt("Run a workflow for implementation"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("spawn a subagent"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("start a workflow"), []);
assert.deepEqual(capabilitiesRequestedByPrompt("create a workflow"), []);

Keep as positives: 用 Subagent 并行检查这三个模块, 子代理了解下项目, 用 Workflow 分阶段实现和审查, plus the existing Delegate this task. / Please parallelize this work. / Could you delegate this task? path. 来多子代理一起讨论 stays a positive via 多子代理 even after / are removed.

tests/extensions/capabilities/index.test.ts (not in the files list):

  • Flip reserved capability words load their groups without action verbs: after h.before("subagent, workflow"), active must stay read/bash/edit/write (no subagent_spawn, no workflow).
  • Remove Use multiple subagents. from the explicit-intent positives (PR regex already misses it; index.test.ts still asserts load).
  • Flip Use subagents to parallelize this task. to no-load, or replace that prompt with Delegate this task. (already locked elsewhere). parallelize this task does not fire when prefixed by Use subagents to.

tests/extensions/capabilities/ui.test.ts (not in the files list):

assert.deepEqual(editor("subagent, workflow").highlighted.render(120), ["subagent, workflow"]);
assert.deepEqual(editor("Subagent 和 Workflow 有什么区别?").highlighted.render(120), ["Subagent 和 Workflow 有什么区别?"]);

Keep highlight on 用 Subagent 检查,再用 Workflow 汇总 and 子代理了解下项目. Add no-color cases for I use subagents in this repo and 开始看子代理代码 so lavender cannot keep authorizing use+name / /.

SETUP/README sentences that would lie after this merge

Docs are unchanged. Runtime would stop treating bare English names as authorization. /openpi-setup status (Capability discovery: explicit) would not lie. These would:

  • SETUP: The case-insensitive English words \subagent` and `workflow` are reserved authorization words: entering either word is sufficient to load its capability group — true today, false after merge (subagent, workflow[]`).
  • SETUP: This makes discussion that contains either English word an intentional opt-in tradeoff — false after merge for Compare Subagent and Workflow; still true for I use subagents in this repo, so discussion is neither uniformly opt-in nor uniformly inert.
  • README TIP: 英文 \subagent` 与 `workflow` 是保留授权词,单独输入也会加载对应能力。`
  • README TIP: 例如 \subagent, workflow` → 同时加载两组` — the advertised-phrase test does not lock this example, so CI will not catch the lie.
  • README TIP: 因为英文名称本身就是授权词,讨论中写出它们也会开闸
  • README TIP: 在交互输入框中,保留词 \Subagent` / `Workflow` … 会使用 … 薰衣草紫` — bare reserved words would no longer highlight.

This PR’s own new file header (references in identifiers, discussion, or comparisons stay inert) would also lie: identifiers/comparisons become inert, discussion (I use subagents, how to use subagents, 接下来讨论子代理设计) still loads.

Rewriting SETUP/README to match that header without removing use+name and / just replaces old false sentences with new ones. Chinese action forms (用子代理检查, 子代理了解下项目, 用工作流编排) stay true and must keep their tests.

CHANGES_REQUESTED still stands. This comment is the missing-test list, not an implementation.

@tt-a1i

tt-a1i commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Missing-test contract (review cannot flip on 341e126)

Pinned: checkout a9b40f0044ee59c360a6077c1c0bdbdbd30da10b, origin/main 0d17f4577fe31315fe6c95370d251bdb4e2413cf (capability-intent.ts blob f7da1d54 identical), this PR head 341e126fac36358ccd08b1c509d39eb16b705e4a. Review still CHANGES_REQUESTED. This is the test list a replacement must lock before review can flip — not another behavior recap. Local contract: docs/research/EXPERIMENT_357_MISSING_TESTS_2026-09-09.md.

341e126 only retargets tests/extensions/shared/capability-intent.test.ts. It does not touch tests/extensions/capabilities/index.test.ts (reserved capability words… at L247–255; Use multiple subagents. at L261), tests/extensions/capabilities/ui.test.ts (subagent, workflow / 「Subagent 和 Workflow 有什么区别?」 at L105–124), SETUP.md L63, or README.md TIP L88–92.

Required cases (intent and the sibling layer named). = no group + no subagent_spawn/workflow*. Color is not evidence.

English reserved words → : subagent, workflow; bare subagent/workflow/Workflows/SUBAGENT; Compare Subagent and Workflow; Subagent 和 Workflow 有什么区别?; git checkout -b subagent-matching; subagent.ts / review PR about subagent; feat/subagent-isolation / extensions/subagents/ / fix/subagent; .github/workflows/ci.yml / workflows/SKILL.md; this is a GitHub Actions workflow.

Chinese / windows → : 我开发对应改代码子代理模块; 开始看子代理代码; 接下来讨论子代理设计; 打开子代理面板; 过来看看子代理. Do not keep 开.{0,8} to save 开个子代理 — that miss is a SETUP policy, not a wider regex. 开发 OpenPI 子代理 stays (window > 8).

True request that this diff drops: Use multiple subagents. must stay delegate + index loads subagent_spawn. use (a )?subagents? cannot host multiple. Do not delete the index positive to go green.

Absence / discussion must not authorize: 聊聊子代理 / 子代理是什么; ordinary prompts with no capability words (fix the flaky test in src/auth.ts); review negative I use subagents in this repo plus how to use subagents / I use workflows in this repo.

Keep fail-closed positives and negations: Delegate this task. / 子代理了解下项目 / 用子代理检查 / 用工作流编排; Do not use subagents. / 如果需要,请使用子代理。. Classifier stays a pure function of prompt (no model-judge). Highlight and before_agent_start keep sharing capabilitiesRequestedByPrompt. Default stays explicit. Load stays monotonic.

CI on this head died at biome format on the intent test; bun test never ran. Format-green is not #303-closed. Do not merge 341e126. No new Issue.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 已修复,明确请求和讨论语句已分离,双线复审无 P0/P1。check 与27项聚焦测试通过;完整 Node 测试1542通过、1跳过、1项未修改的 setup 子进程8秒超时,保留原断言的外部诊断8.2秒通过;130项 Vitest 通过。最新 CI 待确认。此账户推送了修复,仍需另一维护者满足最后推送后的独立批准。

@tt-a1i
tt-a1i merged commit 34fed9b into openpi-dev:main Sep 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

explicit 能力发现把 subagent / workflow 的出现当成授权,dogfood 和粘贴都会误开

2 participants